Great strides have been made recently in the field of artificial intelligence toward developing electronic circuits which emulate higher order functions performed by the human brain and the brains of other animals. These circuits are commonly known as neural networks. They generally take the form of a matrix comprised of a set of horizontal lines which cross and are coupled to a set of vertical lines. The horizontal lines simulate the functions of axons in the cortex of the brain and provide the inputs to the network. The vertical lines simulate the function of dendrites. The vertical lines are terminated at summing devices which replicate the function of the soma, otherwise known as the neuron cell body. Examples of such networks can be found in U.S. Pat. Nos. 4,950,917; 4,906,865; and 4,904,881.
Within a neural network, electrical circuits are employed to model the function of a biological synapse. Collectively, these circuits provide a connection between the horizontal and vertical lines of the network. Individual synapse cells provide a weighted electrical connection between an input and summing element (i.e., a neuron body). The relative strength of the connection often changes during the training or learning process.
In earlier work, electrical synapse cells were implemented using ordinary digital resistors and/or digital-to-analog converters to provide the weighting factor or function. More recently, floating gate devices--which modulate current flow depending on the value of a stored charge--have been employed for this purpose. Examples of semiconductor synapse cells which employ floating gate devices in this manner are found in U.S. Pat. Nos. 4,956,564; and 4,961,002.
Electrical synapse cells may be either analog or digital in nature. For an analog implementation, the weighted sum of input signals is usually computed by summing analog currents proportional to the product of the inputs, with stored weights. In such a cell, considerations of cell size and resolution of the connection weight must be carefully balanced. Furthermore, in those types of networks which employ a pulsed analog input voltage, there is a need to maintain accurate frequency, amplitude, and duty cycle control of the analog input signal. At high frequencies, this is often difficult to achieve.
As will be seen, the present invention discloses a novel synapse cell which operates by transferring packets of charge between associated vertical dendrite lines. The invention has the advantage of inherently providing two-quadrant multiplication of an input vector and a stored weight, and can easily be adapted to four-quadrant multiplication. Moreover, the invented synapse cell benefits from a very low device count and is ideally suited for networks which implement the Sigmoid function as part of the summing process. The invented cell can operate in a continuous analog mode, or alternatively, it can operate digitally. Digital operation is useful for performing pattern recognition tasks and for computing results such as the Hamming distance between two binary patterns. The present invention also obviates the need for accurate control of input amplitude and duty cycle. Another advantage is that power dissipation can be much lower than current mode computation.